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ABSTRACT 


Security of a Forward Operating Base (FOB) is of high interest and operational 
importance to the U.S. military and allied forces. The Situational Awareness for 
Surveillance and Interdiction Operations (SASIO) model simulates the operational 
tasking of a single Unmanned Aerial Vehicle (UAV) and a ground-based interceptor that 
are designed to search, identify, and intercept potential hostile targets prior to reaching 
the FOB. This thesis explains the SASIO model and its implementation in JAVA. This 
theoretical model leverages Design of Experiments (DOE), which varies multiple 
characteristics of the system to explore insights for the tactical employment of UAV and 
interceptor to combat potential hostile actions against a predefined area of interest. 
Designed screening simulation experiments identifies influential factors to provide 
guidance for tactical employment of Blue Force assets, as well as provide alternative 
means to influence Red force behavior in a beneficial manner. This thesis analyzed the 
effects of the influential factors with respect to the percentage of threats interdicted, time 
to acquire threats, and mean distance away from the FOB that the threats were 
interdicted. Through analytical techniques, a quantifiable measure of the employment 


strategy for the UAV and ground-based interceptor was achieved. 
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EXECUTIVE SUMMARY 


Security of a Forward Operating Base (FOB) is of high interest and operational 
importance to the U.S. military and allied forces. The Situational Awareness for 
Surveillance and Interdiction Operations (SASIO) model simulates the operational 
tasking of a single Unmanned Aerial Vehicle (UAV) and a ground-based interceptor that 
is designed to search, identify, and intercept potential hostile targets prior to reaching the 
FOB. This thesis explains the SASIO model and its implementation in JAVA. The goal 
of this thesis is to provide insight into influential factors that lead to developing the 
matrix pertaining to the percentage of threats interdicted, time to acquire the threats and 


distance away from the FOB that the threats were interdicted. 


Insights and Recommendations 


Utilizing analytic techniques, the thesis explored the employment of Blue Force 
assets, as well as Blue Force action that is external to their asset employment strategies. 
The sensor characteristics of the UAV minimally impacted the response variables studied 
for this scenario. However, the UAV as a surveillance asset did provide the ground-based 
interceptors an increased detection range allowing for a marginal increase in interdicting 


the threats. 


Utilizing a single UAV and ground-based interceptor, the actions of whether the 
ground-based interceptor patrolled or remained stationary at the FOB dramatically 
influenced the percentage of threats interdicted, as well as the mean distance away from 
the FOB. If the ground-based interceptor patrolled, the percentage of threats interdicted 
decreased by 53%, while the mean distance away from the FOB for interdiction increased 
by 11%. Therefore, through analytical techniques, a quantifiable measure of the 


employment strategy for the ground-based interceptor was achieved. 


The most influential factor with respect to all of the responses is the enemy’s 


speed as they traverse towards the FOB. As the speed of the enemy increased, each 


xiii 


response degraded. Therefore, if external checkpoints are provide by indigenous 
personnel to reduce the enemy’s traversal speed, then the Blue Force can more effectively 


interdict the threat. 


Scenario and Concepts of Operations 


The probability and stochastic models discussed in this thesis include multiple 
moving neutral and threat ground objects and a surveillance and interdiction force 
comprising a single UAV for surveillance and identification and a single ground-based 
interceptor designed to intercept and clear any hostile objects. Upon initialization of the 
scenario, the threats and neutral objects arrive in the area of interest randomly and 
traverse towards the FOB. The goal of the Blue Force is to detect and identify the threats, 
issue commands to the ground interdictor, and clear the threats prior to reaching the FOB. 
Once the UAV detects and identifies (with some identification error) the threats, then the 


interceptor’s responsibility is to traverse towards the threat in order to clear that threat. 
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I. INTRODUCTION 


A. PROBLEM/ NAVY INTEREST 


Surveillance and Interdiction (SI) problems are of great interest to the United 
States military due to the necessity to obtain pertinent information of potential hostile 
threats and to neutralize those threats upon their correct identification. The military 
utilizes numerous assets such as, but not limited to: Unmanned Aerial Vehicles (UAVs), 
Ground Based Operational Surveillance Systems (GBOSS), and satellite imagery to aid 
in categorizing the entities’ actions as threats or not. Upon detection and identification of 
possible threats, the military must take action to minimize and/or perturb the 
aforementioned threats to maintain security within the regions of operation. Linking the 
surveillance and interdiction efforts together is an operational hurdle that must be 


overcome to ensure the safety of our military forces. 


The attack on the USS Cole (United States Department of Defense, 2001) may be 
classified as an SI problem, and it illustrates the necessity to process relevant information 
in a rapid manner. Given the atmosphere surrounding this event, proper patrolling of the 
surrounding waters, as well as inspection of any approaching vessels prior to allowing the 
vessel to tie up to the ship, may have thwarted this attack. Another attack that falls within 
this class of problems is the Suicide Vehicle-borne improvised explosive device (VBIED) 
attacks on an Iraqi Army checkpoint (Multi-National Corps—Iraq, 2008). Quick Reaction 
Force (QRF) units and checkpoints were established to prevent such an attack; however, 
the actions by embassy personnel were not able to properly identify and detect the 
VBIED. The overall problem, as noted above, is the fact that although information and 
data surrounding an operational environment may be available, the ability to take action 
to mitigate such attacks requires relatively quick analysis that produces feasible and 
desired results. In this context, for example, “the true operational art behind that strike 
[airstrike that killed Zarqawi] was a multidisciplined intelligence, surveillance, and 
reconnaissance endeavor with agile special operating forces ... resulting in a find-fix- 


finish operation” (Cantrell et al. 2008). 


One such instance of the necessity for find-fix-finish operations is for the 
installation’s protection. Suppose the military is operating within the confines of a 
Forward Operating Base (FOB) within a region. The security of this FOB is paramount 
due to the military operations that take place in the region. Military scouts, such as 
UAVs, may be flown for surveillance purposes to allow the FOB to extend its detection 
and identification range to ensure an adequate standoff distance from any threats that may 
try to attack and/or infiltrate the base. The data obtained from the UAV can either be 
automated via tracking and identification algorithms, which allows the UAV to perform 
its operations without the need of a human to make complex decisions, or employ the 
current “human-in-the-loop” process, which requires a human to process mass data in 
either live-feed video or still imagery. Based on the data, the commander must make 


complex decisions to counteract actions that may be threatening the installation. 


The decision to engage an enemy must be made based on quick-turn analysis of 
surveillance data such that the entity is ideally cleared outside of a predefined standoff 
distance away from the installation to ensure minimal loss to life and property. Current 
doctrine requires action on any potential threat; this ensures that all entities are cleared 
through interdiction. However, following this doctrine, manpower is not utilized in an 
efficient manner, and if limited resources exist, then clearing all entities may not be 
possible. This thesis illustrates how the SASIO simulation model can be used as an 
analysis tool to provide commanders with insight on employment techniques to counter 
potential threats given limited surveillance and interdiction resources to protect a FOB (or 


any other location that requires protection). 
B. BENEFITS OF STUDY 


Surveillance and Interdiction problems appear in many forms. The military is 
relatively good at gathering the data from surveillance assets, but linking potential flags 
from one data set to another to obtain a course of action during military operations is 
relatively slow. During military operations, Combatant Commanders are either not given 
concise information, which may or may not require immediate action, or they are 


presented with data requiring actions that ultimately exhaust the assets without 


2 


investigating all of the potential threats or options. Understanding and acting on only 
pertinent intelligence in a timely manner can alleviate exhaustion of assets, as well as 
minimize the risk associated with performing extraneous interception missions for low- 
threat targets. This thesis explains the Situational Awareness for Surveillance and 
Interdiction Operations (SASIO) model and how Combatant Commanders can use this 
model in support of relevant SI settings. Situational awareness, as defined for this thesis, 
represents the knowledge and understanding of the environment and the actors within that 
environment. Use of the SASIO model supports either real-time employment strategies 
(decision support) or robust design strategies (analysis tool) to maximize the employment 
of surveillance and interceptor assets. The pertinent intelligence gathered from the 
employment of such assets could enable more effective interdiction of threats as they 


may arise or before they engage. 


The SASIO model was developed to provide a decision support tool based on 
currently available and forward-deployed assets. SASIO may utilize current 
characteristics of surveillance and QRF assets to provide an employment strategy via 
multiple simulation iterations, as well as provide real-time support through simulation 
iterations based on the ever-changing theater. The SASIO model can support many 
classes of SI contexts ranging from the aforementioned FOB protection, border 
protection, counter drug operations and Maritime operations with regards to anti-pirating 


operations. These types of problems share common components that comprise any SI 


missions: 

® Utilize surveillance assets to patrol the area to identify threats pertaining 
to the security of the area of operation 

® Detect and classify threats and neutral actors based on the intelligence 
gathered from the surveillance asset 

e Transmit the information obtained by the surveillance asset to interdiction 
assets 

e Interdiction assets can either investigate and/or interdict the actor based on 


the transmitted report or wait for additional reports 


Utilizing this framework, while combining the effects of probabilistic sensing and 


motion by Blue Force assets, the evolution of this model can provide Commanders with a 


decision support tool. SASIO links the gathering of information to the execution of 
actions that must be performed to ensure security of an area through recommendations 


and potential outcomes of a failed action. 


In support of such a tool, SASIO provides insight into the factors of relevance for 
performing said tasks, such as: speed of each asset (UAV, QRF), sensor characteristics 
(false positive and missed detection) rates, when the QRF takes action and also aid in 
determining a secure location for a FOB within a hostile region. SASIO may be iterated 
through with varying factors relatively quickly, which could allow an analyst to provide 
asset employment recommendations based on the current operational environment being 
studied. Combining this fact and performing sensitivity analysis on these factors, the 
potential shortcomings of current technology and doctrine (e.g., Tactics, Techniques and 
Procedures) may be evaluated to provide meaningful insight in the context of SI 


problems, and revised as necessary. 


This thesis makes use of the SASIO model to explore the effectiveness of a single 
UAV and a single QRF with regards to the protection of a FOB. Numerous factors are 
examined in order to analyze their significance in the proper employment of these forces 
to combat a potential threat to the FOB. This threat is characterized as a vehicle-borne 
improvised explosive devise that can damage the FOB even if the threat does not reach 
the FOB. As the distance from the FOB to the interdicted threat is increased, then the 
destructive capability of the threat is decreased. The scenario makes use of the 
topographical constraints and road network found at Camp Roberts Army National Guard 


installation in California. 


This work is motivated by ongoing field experimentation efforts via the Naval 
Postgraduate School and the United States Special Operations Command (USSOCOM) 
Field Experimentation Cooperative program (Bordetsky et al., 2009). The current 
experimental scenario includes a forward-deployed, 12-man team designed to protect 
their command installation (Special Operations team house) while performing 
intelligence, surveillance and reconnaissance (ISR) operations via UAV assets. Through 
use of limited resources, primarily minimal surveillance assets, employment tactics for 


the UAV based on varying theater configurations (SOF team house placements) are 
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evaluated. Based on time constraints (e.g., long lead time required to perform a single 
live experiment), the SASIO model can be used in conjunction with current field 
experiments to provide analysis based on current tactics utilized, as well as provide 
insight into alternative employment strategies to maximize the protection of an 


installation. 


The SASIO simulation engine incorporates a modular type framework. The user 
may select any currently built-in motion or interdiction models, or may create their own 
to test out new employment techniques for both the surveillance and interdiction assets 
alike. This feature is useful given that an employment for one theater may not be efficient 
or appropriate for another. This modularity ensures that the user is not limited to a single 


technique, but is able to evaluate a multitude of strategies to combat an evolving enemy. 
C. RELATED WORK 


There have been recent research efforts with respect to the employment of UAVs 
for ISR type missions. Prior to performing ISR tasks, the underlying class of problem 
must be understood. Mission planning and employment must take into account aspects of 
classical search modeling as it applies to the imperfect sensor characteristics of the UAV. 
Washburn (2002) studied the propagation of errors in a simulation due to an imperfect 
sensor. Berner (2004) studied sensor characteristic of varying UAVs to obtain situational 
awareness for protection of a high value target while Raffetto (2004) focused primarily 
on the factors of airspeed and loiter times for the UAV. Berner’s (2004) study analyzed 
how a broad area search UAV with imperfect sensor characteristics may be used in 
conjunction with a better detection and identification UAV with a relatively lower 


detection range to increase the coverage surrounding a particular mission region. 


Once situational awareness is achieved for a region, the ability to allocate a QRF 
and proper employment of alternate surveillance assets must be performed to acquire 
targets within a short time after identification. Chung et al. (2009) formulated a Mixed- 
Integer program utilizing Bayesian updates to perform the task of allocating surveillance 
and QRF assets to rapidly identify potential targets. Similarly, this thesis focuses on the 
tactical uses for each asset. Other methods to rapidly interdict potential targets utilize a 

5 


Subject Matter Expert (SME) to manually interject proposed enemy trajectories based on 
experience in the region. Kress et al. (2007) demonstrated significant improvements in 
search performance through the implementation of a “two-stage stochastic Integer Linear 
program” in conjunction with SME inputs. Although this is an interesting improvement 
to the SI class of problems (consisting of an increase in 50% detection opportunities 
based on a preplanned generated UAV routing provided by the SME), the paper does not 
address variations in factors as they pertain to surveillance, such as: sensor 
characteristics, sensor footprint, sensor loiter time and platform speed. SASIO seeks to 
investigate such factors, as well as others, to obtain a more robust design for coordinated 


employment of each type of agent. 


Much of the past research has focused on the surveillance abilities of the UAVs. 
Upon a UAV observation, the UAV transmit the location of the threat to the QRF with 
some time delay due to transmittal distance. With this time delay, an error in the exact 
location of the threat arises and the location must be offset to account for this error to 
provide a successful interdiction. Bertuccelli et al. (2006) illustrated the importance of 
taking into account variability in information due to time delays for UAV search and 
ground interceptions. The author used Uncertain Probability Maps (UPMs), which 
accounted for poor knowledge and propagations of the target location throughout the 
environment. Through recursive calls of a distribution of the targets’ trajectories, the 
maps are generated and updated for efficient search. In this thesis, UAV routes are 
heuristics based on a probability map representation of situational awareness that is 


updated with real time observations. 


For the purposes of this thesis, SASIO performs the tasks of SI based on a single 
UAV for surveillance and a single QRF. Other past works have studied the case where 
multiple assets were utilized. Yan et al. showed UAV performance of multiple tasks 
based on their capabilities such as: searching, confirming, attacking, and performing 
Battle Damage Assessment (BDA) (Yan et al., 2003). The authors used deterministic 
Discrete Event Simulation (DES), continuing efforts until all targets were eradicated. 
This model illustrated that there exists a point in which an increase in the number of 


UAVs showed diminishing returns. Also, an active search algorithm for the UAVs was 
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more effective than a random search in eradication of targets. In contrast, the SASIO 
project utilizes a time-stepped simulation, integrate imperfect observations with Bayesian 


updates, and vary numerous factors pertaining to Red and Blue Force assets and theater. 
D. THESIS ORGANIZATION 


The main contributions of this thesis are the development of the SASIO model as 
well as the analysis and resulting insights for tactical employment of Blue Force assets in 
an SI context. Chapter II outlines the basic elements used in constructing the SASIO 
model. In Chapter III, the thesis delves into the details of the model while highlighting 
potential variations in the model to explore alternate scenarios. Chapter IV discusses the 
scenario analysis and presents the reader with the results of the analysis, as well as the 
operational and tactical insights that may be obtained from the analysis. Chapter V 
provides a summary of this research and the conclusions as well as present avenues for 


future research. 
E. SCENARIO 


Within the area of interest (AOI), the military has created a FOB where military 
operations for the immediate region based. There are three roads that we consider three 
entities entering this area, one of which is a threat, while the other two are neutral. Each 
entity enters the AOI and traverses the road toward the FOB at a constant speed. 

The assets in place to protect the FOB (Blue Force) consist of a single UAV for 
surveillance and a single QRF to interdict and clear the threat. The UAV performs 
surveillance along fixed radii paths from the FOB. The purpose of the UAV is to give the 
Blue Force the ability to identify a threat while at an increased standoff distance from the 
FOB. Upon a detection and identification, the UAV issues a report for the QRF to 
interdict the threat. 

The QRF performs a patrol surrounding the FOB at a distance of 250 m along the 
road network. The QRF has the ability to perform surveillance, as the patrol is occurring 
and interdict threats that are within this surveillance range. Therefore, the QRF provides a 


second line of defense to the FOB. 


F. ASSUMPTIONS 


Theater 

Road Network — The road network is a directed graph from entry nodes to the 
AOI towards the FOB. 

Node Separation — Each node is 250 m apart. 

Theater Map - Each node has a probability of a threat in that node and continues 
to be updated as observations are made by the Blue Force assets via a Bayesian updating 
process. 

Objects 

Expected Arrival Time — The objects have a minimum inter-arrival time of one 
unit. 

Traversal Speed — Threats and neutrals traverse at a fixed speed (unknown to the 
Blue Force). 

Traversal Direction — All objects traverse towards the FOB. 

Agents (UAV) 

Surveillance Path — The UAV flies a fixed waypoint paths at varying distances 
from the FOB. 

Detection/Identification — The UAV has perfect detection and imperfect 
identification. 

Report Issue — The UAV issues a report upon (possibly erroneous) positive threat 
identification. 

Agents (QRF) 

Patrol — The QRF performs a patrol of either a single node waypoint path away 
from the FOB or remains stationary at the FOB. 

Clearing — Upon interdiction of a threat, the QRF detains the threat and clear the 
threat after the appropriate time has elapsed. 

Visible Cells — The Blue interdiction force has the ability to perform surveillance 
within its current cell as well as adjacent cells based on the users input. 


Detection/Identification — The QRF has perfect detection and identification 


I. MODEL FORMULATION 


A. OVERVIEW 


This chapter describes the development of the theater, scenario and taskforce. The 
theater governs the region in which the entities interact. For this thesis, the theater is 
specified as a region patterned after Camp Roberts Army National Guard facility in 
California. The scenario is synonymous with the objects not characterized as Blue Force. 
These objects may be categorized in the form of neutral or threat objects. The last portion 
of the model is the taskforce entities, which includes all Blue Force assets, surveillance 


and QRF alike. 
B. DESCRIPTION OF THEATER 


The AOI for this scenario is at Camp Roberts, and was chosen due to the ability to 
utilize aerial and ground based assets to calibrate the model. The chosen roads form a 
“Y” shaped network with the FOB located where the three roads intersect. Each road is 
discretized into nodes separated 250 meters apart from each other and the labeling of the 
nodes is arbitrary, though once labeled, each node is maintained and known to all Blue 
Force entities within the simulation representing a common operational picture (COP). 


The scenario theater is depicted in Figure 1. 






@ Entry Nodes 


() Intermediate 
Nodes 


> Fos 


Figure 1: Camp Roberts, CA: FOB located in center and depicted with blue diamond. 
Entry nodes are illustrated as red circles, while intermediate nodes are in white. Each 
node is separated in 250-meter increments. 


The graph representing the road network is an undirected graph linking each node 
to the adjacent nodes. This allows for objects, Red and Blue ground elements alike, to 
transit throughout the area by moving from a single node to its adjacent node. The 
southern road has only 1.5 km in total distance from entrance to the protected FOB 
location, while the other northern routes are 3 km in total distance. This was determined 
based on airspace constraints within Camp Roberts. The separation of 250 m from one 
node to the next was chosen due to the nominal sensor footprint of the aerial asset 
utilized. In other words, the asset, while flying at its nominal altitude, provides a 250 m 


wide footprint, thus allowing thorough observation of a single node during each pass. 
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C. DESCRIPTION OF SCENARIO 


1. Arrival Model of Vehicles 


Both neutral and target type vehicles enter the road network via the entry nodes 
and travel towards the FOB. Assume that any object traversing the network away from 
the FOB pose no threat to the installation and can be disregarded. Each object’s arrival is 
assumed independent and the rate associated with their inter-arrival times to begin their 
transit down a road is denoted by parameter rate 1 (an event per unit time). Consider 
time to be discretized in At increments, utilizing the underlying properties of the 
geometric distribution as the arrival process; the probability that the target arrives at time 
t is AAt. Each arrival is independent from the other and memoryless due to the geometric 


distribution. 


Suppose that the total number of vehicles, N, will not exceed three vehicles on the 
road network; thus, N = 3. This can be assumed due to a coordinated checkpoint system 
in the road network to ensure that the network is not overwhelmed with traffic. The 
scenario evaluates the arrival of the objects under three distinct instances of inter-arrival 


rate: Ay, A2, 43 Where 1 =A)<A2< Az. 


When stating that the arrival of objects has a one-time step inter-arrival time, this 
simply states that objects arrive with one time unit separation. The investigation of Blue’s 
ability to properly interdict the target while observing all objects, threat and neutral alike, 
is a difficult task. Analyzing the one inter-arrival instance provides a worse case basis 
against which all later scenarios may be evaluated. Then the thesis will explore the best 


Blue Force case with a large inter-arrival time. 


An Intermediate inter-arrival time (A2) is simulated to ensure that nominal traffic 
patterns are explored. Through simulation and analysis, this inter-arrival time provides 
insight into a more realistic scenario setting, since the objects will not be all arriving at a 
single time, nor arriving in such a manner that only a single object is in the scenario 


during any given time. 
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The largest inter-arrival time (A3) utilized for this thesis is setting the inter-arrival 
time to the time required to traverse the longest road by any single object. Under this rule, 
the simulation may be thought of as only exploring a single object’s motion from the 
entry into the theater to the destination at any given time. Since there is only a single 
object to be identified and interdicted, an upper bound with respect to the Blue Forces’ 


ability to perform its operations will be analyzed. 
2. Random Selection of Roads 


Given the arrival rate of vehicles to the road network, the avenue of approach, 1.e., 
the road traversed by each vehicle, is also assumed a random component of the scenario. 
The theater consists of three roads that lead to the FOB location. The three roads have a 
distinct entrance node for the enemy to begin its transit. Supposing that there exist three 
objects, one being a true target, the initialization of where the objects begin may be 


simulated in the following ways: 


e The objects begin on each of the three roads, such that each road contains 
only one object 


° The choice of roads is independent from object to object 


The first proposed concept for object placement is realized by drawing a random 
number based on the number of roads that exist. Once the number is drawn, the road is 
assigned to an object and is then discarded from the possible choices of roads. This 
process is repeated until all roads have been assigned to the objects. For example, 
consider the Camp Roberts’ network where entry nodes are labeled 7, 19, 31, as shown in 
Figure 1. Utilizing the idea of assigning a road to an object and then discarding that road 


as a possible choice ensures that the roads are not duplicated from object to object. 


R~U(0,N); where N is number of entry nodes 


Next iteration: 


R~U(0,N-—1); where N is number of entry nodes 
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The second concept allows each object access to all of the entry nodes as their 
initial starting location. A random number is drawn from the list of possible starting 
locations. Based on the random number, the node is chosen and then assigned to an 
object. The iteration of this process is continued until all objects have an assigned entry 


node. 


R~U(0,N); where N is number of entry nodes 


The former method of road selection by the objects depends on the objects that 
have already arrived. This dependence simplifies the scenario because once an object has 
been observed on a single road and interdicted, then that road would not need to be 
observed again. Although this is an assumption of the arrival process, the simulation does 
not account for this dependence and continues to search for objects regardless of whether 


an object was found on a single road or not. 


The road selection method that allows for multiple objects to arrive on a single 
road is independent from one road selection to the next. This type of problem is more 
complex for interdiction operations due to the fact that objects may arrive on the same 
roads shortly after other prior arrivals. Therefore, if an observation is made and the 
surveillance asset continues on to the next road, the remaining objects may traverse the 
road undetected by the surveillance asset. Thus, the only chance that the object is 
interdicted prior to reaching the FOB rests with the surveillance capabilities of the QRF. 
Supposing that the QRF does not have the ability to look out past the current location and 
the QRF is not performing a patrol, then all the objects that leak through the surveillance 
asset will reach the FOB. Comparison of these two approaches highlights any advantage 
of considering the more difficult independent case. Modeling the interdiction failure and 
the percentage of objects that reach the FOB under this premise provides insight into 


operational tactics and utilization of the surveillance and interdiction assets. 
3. Object Motion Model 


When developing the SASIO model, the discrete time step used to measure the 


progression of entities represents an arbitrary unit of time. The time step utilized 
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throughout the model is scaled in such a way that all objects, neutral or threat, are 
assumed to not traverse more than a single cell in a given unit of time. This allows for an 
observation to be made in any given cell such that the object is either present or not due 
to motion through the cell and not by “jumping over” a cell. This assumption is relevant 
and is utilized when the surveillance and QRF assets perform observations which later 
feed into an updated probability map which illustrate the probabilities of where an object 


is within the theater. 


Assume that each vehicle approaches the FOB with a constant, deterministic 
speed, denoted by v. For the discretized road network, the vehicles’ speeds are converted 
from a continuous velocity to that which is discrete which corresponds to the number of 
cells traversed in a given time step. For example, v = 1 corresponds to a traversal of one 
nodes in a single time step, where as v = 0.25 is analogous to a slower vehicle requiring 
four time steps to transit to an adjacent node. Upon starting the simulation, the objects 
speeds are set to one of the following: 1 (the objects move every time step), 2 (the objects 
move in two time steps, i.e., v = 0.50), and 3 (three time steps to move, i.e., v = 0.33). 


The justification for modeling object movement in this way follows. 


Given the theater is constructed so all object traffic tends to flow from an entry 
node towards the FOB, a directed graph may be constructed based on the adjacency list 
of the theater. The pseudo code in Figure 2 represents the object motion model for the 
objects. From this implementation, a visual representation of the motion model, as in 
Figure 3, illustrates the pseudo code execution of an object moving in 3 time steps, as it 


pertains to the Camp Roberts’ scenario. 
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Input: Directed graph from Theater 
Object's speed 


-Directed Graph, 
sg’ 
-5et Counter = Object speed 
Algorithm 
1 While(true] 
2 «Decrement Counter 
3 lfi(Counter == 0) 
4 -Determine adjacent node 
5 -Transit 
6 -Reset Counter to Object's speed 
? End 





Figure 2: Pseudo code representing Object Motion Model implementation 





Figure 3: Object motion model implementation for an object that traverses from 
node to node in three time steps 
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The above models pertain to the objects defined in the scenario. Future studies 
may investigate more sophisticated constructions facilitated by the modular design of the 


SASIO framework. 


D. DESCRIPTION OF THE TASK FORCE 


1. Platform 


a. Surveillance Asset: Unmanned Aerial Vehicle 


The Unmanned Aerial Vehicle (UAV) performs surveillance operations as 
dictated by the operator. For this scenario, the UAV performs a patrol pattern of given 
radius from the FOB. Based on its operating speed, the UAV flies each node in its given 
waypoint list, performing observations and identifications upon arrival to a node, 
loitering briefly, and continuing its patrol route. Such a pattern can be easily 
accommodated by current waypoint-based UAV auto-pilot, enabling operators to focus 


on examining down linked imagery. 
b. Interdiction Asset: Quick Reaction Force 


The Quick Reaction Force is a mounted vehicle asset whose primary role 
is the security of the FOB. This asset performs surveillance through patrol routes, as well 
as interdict potential threats that are obtained from the surveillance asset. While 
performing patrols, the QRF performs a predefined patrol pattern until the surveillance 
asset makes an interdiction request. The QRF transitions to an interdict mode and 
services the request by traversing towards the location where the request originated. Once 
an interdiction is made, the QRF clears the object and then returns back to the FOB to 
continue its patrol. This process of patrolling, interdicting and returning back to patrolling 
provides protection for the FOB through surveillance and interdicts the threat while 


maintaining a standoff range from the FOB. 
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2. Probabilistic Modeling of Situational Awareness 


a. Probability Maps 


The SASIO simulation utilizes a dynamic probability map to govern a 
actions by the QRF. A probability map, as defined in the context of this thesis, is an array 
of probabilities for each node within the theater, which allows the user to perceive the 
likelihoods of where an enemy may be located. The user initializes the simulation with 
what is believed to be the location of the threat or the probability that the threat may 


traverse a given set of node based on intelligence gathered for the area. 


The probability map representation used in this thesis is an important 
aspect of SI problems. The probability map’s primary function is to resolve whether a 
query and/or interdiction of an object must occur by the QRF. The QRF may be 
performing surveillance operations in conjunction with the UAV; however, once the 
Commander deems that there is a threat object along a road with high confidence, then 
the QRF transitions to the interdiction mode to pursue such a threat. This transition 
directly relies on the probability map rapidly evolving as observations are made by the 
UAV. If no intelligence is obtained for the AOI, then the user can define a uniform 
probability along the entire theater as follows 

Let X € {1, 2, ..,C} be defined as a RV of the location of the object 


where C is the number max number of nodes in the theater 
— P[X,]= probability that a threat is located in cell C 


PLX,J~U(O.D)= PIX1= = 


As the simulation progresses and the surveillance asset makes 
observations, the probability map is updated based on these observations according to the 
observation models described in the following section. Suppose that the surveillance asset 
performs an observation in cell five and the surveillance asset does not observe a threat, 
then the probability that a threat is in cell five is decreased for the probability map. The 
converse is also true. The way in which the probability map incorporates imperfect 
observations is through a Bayesian update. 
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b. Bayesian Updating 


Each node is updated independent from the other cells. Therefore, if an 
observation is made in one cell, then that cell alone will be updated. Using this method to 
update the cells is used over a joint cell updating because as more UAVs are used, the 
updating process becomes more difficult. Given that the cells’ identifications are 
imperfect, the single cell updating is more plausible because after an observation and 


identification is taken, then only that cell should be updated. 


The detection process for the UAV relies its sensor characteristics. 
Assuming that the UAV is equipped with perfect detection sensors, and the simulation 
was run holding all other parameters constant, then the result of this simulation run would 
be the best-case scenario. Suppose that the UAV is performing surveillance in accordance 
with a user-defined route. As the UAV observes a cell, the UAV processes whether an 
object is present or not with 100 percent confidence due to the perfect sensor suite. 
Therefore only a single “look” is required to verify if an object is present or not. Once an 
object is identified in the cell, then the classification of whether that object is a target or 
neutral must occur. Therefore, if the UAV could detect and classify the objects in the 
AOI with perfect confidence, then an upper bound on the problem is defined assuming 
that all other parameters are held constant from an execution of the simulation to the next. 
Since technological advance have not been made to create the “perfect-sensor package,” 
this thesis explores Bayesian updating of the probability map to classify objects entering 
into the AOI given an imperfect sensor package. 

(1). Detection. The SASIO simulation models the detection 
process made by the surveillance and interdiction assets as a perfect sensor. Therefore, if 
either asset observes an object in the cell that surveillance is being performed, then the 
asset detects that object. The SASIO framework does allow for imperfect detection, but 


for the purposes of this scenario, perfect detection is modeled. 


The basis for the perfect sensor for the UAV and QRF is due to the 
verification runs that were performed at Camp Roberts in August of 2009. Utilizing a 


UAV for surveillance and observing the live-feed broadcasted back to the user, detection 
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of an object was easily achieved by the operator. Therefore, the process of perfect 
detection was modeled in the SASIO simulation. 

(2). Identification. Since imperfect identification is modeled in 
the SASIO simulation, some meaning of an imperfect sensor must be understood. There 
are essentially four different cases for probability of identification that may occur when 
an observation is performed. Prior to stating the four cases, some nomenclature for this 


derivation must be defined. 


Assume that a positive detection occurs in cell c; otherwise, no 


identification takes place 


Let Z,. be a Bernoulli random variable describing the true identity 
as defined below 


z 4 0, object type is a neutral 
: 1, object type is a threat 


Let W. be a Bernoulli random variable describing the observation, 


as defined below 


0, observation "says" object is a neutral 
l, observation "says" object is a threat 


There are four distinct cases that an identification can take on: the 
probability of a correct negative identification “1l-y” (eqn. 2.1) (correctly observing no 
threat present), the probability of a false positive detection “y” (eqn. 2.2) (saying the 
object is a threat when its not), the probability of a missed detection “p” (eqn. 2.3) 
(saying the object is a neutral when it is a threat), and finally the probability of a correctly 
identifying a threat “1-p” (eqn. 2.4). 
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Let Z, = 0 represent an object of type neutral is detected in cell c 


W. = 0 represent an identification "says" object is a neutral in cell c 
RW, = 01Z, = 021-7 (eqn. 2.1) 

W. = | represent an identification "says" object is a threat in cell c 
KW, = 112, = 0)ey¥ (eqn. 2.2) 

Let Z, = 1 represent an object of type threat is detected in cell c 

W. = 0 represent an identification "says" object is a neutral in cell c 
PW, = 012, = lFp (eqn. 2.3) 

W. = | represent an identification "says" object is a threat in cell c 
PW, = 11Z, = 1)=1-p (eqn. 2.4) 


So, now that identification can be made utilizing an imperfect 
sensor, the probability of an object being a threat based on prior knowledge of that 
location must be also taken into account. Using this method of prior knowledge of the 
threat location in conjunction with observations being made by the surveillance asset will 
either increase the probability that a threat is in the observed cell or decrease that 


probability. 


SASIO assumes an independent cell Bayesian update. This method 
assumes that given an observation in a single cell; the probability in that cell alone will 
either increase or decrease based on that observation. Other methods could be utilized, 
such as an update method, considering the joint distribution in which e.g., a positive 
observation in a single cell is made, then the probability that the object in the other cells 
decrease. This method becomes more complex if more than a single object exists in the 


AOL. 
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The process of taking observations and updating the probability of 
where an object is located is an iterative process. Thus, the prior probability is 
continually updated as observations are performed. This updating is through utilizing 
Bayes rule (eqn. 2.5) 

_ PIW, =w|Z, =z]* P[Z, = 1] 


P[Z, =z|W, =w] PW owl 





(eqn. 2.5) 


The left hand side is the posterior probability that an object is a 
threat or neutral given that an observation occurs in cell c. Since in the presented model, 
we utilize the independent cell-updating rule, updating any cell other than the cell that 
observation is being taken in does not need to be performed. This decoupling allows for 
efficient computations, linear in the number of observations vice number of cells. On the 
right hand side of the equation, the numerator represents the probability of an observation 
multiplied by the prior belief of the location of the target. The denominator is the 
marginal probability of all observations. As an example, suppose that the threat is 
observed in cell c and that the object is a threat (i.e., Z=1 | W.= 1). Then the following 


result for cell c in the probability map occurs: 


P[W. =11Z, =11]* P[Z, =1) 


Mz. SW <j ee 
P[W. =11Z, =1]* P[Z, =1] + P[W. =11Z, =0]*(1— P[Z, =1)) 


(eqn. 2.6) 


Substituting eqn. 2.4 for the numerator and expanding the denominator is as follows: 


(1- p)* PIZ, = 1 


P[Z, =11W. =1]= : 
(1— p)* P[Z, =1] + y*(1—P[Z, =1)) 





(eqn, 2.7) 


This iterative process occurs with each observation by the UAV. As the UAV performs 
more and more observation, the probability in that cell continues to change. Supposing 
that the UAV contains a perfect sensor package, y and p are equal to zero, that is, no 


errors in identification, then the equation simplifies to the following: 
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P[Z, =11W. =1]= (1)-prior probability in cell c 





: = : SOT =! (eqn. 2.8) 
(1)-prior probability + 0-(1 — prior probability in cell c) 


Therefore, if there were an ability to create a “perfect sensor,” then only a single positive 
identification would be needed to locate the object. However, the costs to create an 
almost perfect sensor may be so extreme that utilizing more surveillance assets to obtain 
more observations may be more effective. Understanding the effect of improving sensors 


and the implied tradeoffs requires a sensitivity analysis, which this thesis aims to provide. 
Cc. Observation Modeling for the Quick Reaction Force 


The QRF can perform observations the same manner as the surveillance 
asset. If the QRF has the ability to observe nodes that are adjacent to its current location, 
then observations may be performed on these locations. This may be thought of as the 
QRF having a person peering through binoculars as the driver transits the patrol route. 
The lookout may see a single node away (250 m based on the theater) or multiple nodes 


away depending on the line of sight and focal range of the binoculars. 


SASIO takes advantage of a lookout when the QRF is in a search mode. 
However, the difference between the QRF and the UAV asset is there are no detection 
and identification errors associated with the QRF, while the UAV has identification 
errors. Thus, Bayesian updating updates the probability map based on the UAV’s 
observations and aids the QRF as a decision-support tool dictating when an interdiction 


must be performed. 
d. Report List Generation 


The data obtained from observations that result in a positive identification 
must be relayed to the Commander for review and/or action. This information is 
transmitted in a report list format. A report list is a list of all reports made by the 
surveillance asset. The report includes pertinent information regarding the observation 
such as: the time that the observation occurred, the location of the observation, which 


surveillance asset made the report and any description of the suspected target. As with a 
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real SI operation in which observations and reports are sent to the Commander, this 
report list process is emulated in the simulation. Action to either interdict or query the 
objects on the report list represents a Commander sending out a QRF to investigate a 
potential threat. SASIO uses the report list as a tool to transition the QRF from a 
searching phase to an interdiction phase and may do so once the report is obtained. Once 
the report is cleared by the QRF, the report is removed from the list. Current SASIO 


model implementation services the reports in a First-in, First-out (FIFO) type queue. 


3. Trajectory Modeling 


a. Unmanned Aerial Vehicle 


The UAV’s patrol path is not geographically constrained since its flight 
path is not limited (other than by airspace restrictions) for this scenario. Therefore, the 
flight paths that will be investigated within this thesis will be of varying radii of 250 m, 
500 m, and 1000 m away from the FOB. Varying the radius of the flight path can either 
aid the Commander in interdicting threats by detecting and observing the objects at a 
longer range; however, it may also hinder the operations due to the increased leg 


distances from one waypoint to the next. 
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Figure 4: Varying radii flight paths for the UAV with predefined flight profile 
(orange) based on 750 m radius from the FOB 


Let WP be a list of waypoints that are desired for surveillance operations. 
Each waypoint distance is calculated based on Latitude and Longitude coordinates from 
the current node to the next node in the list. Assuming that the UAV flies at a calibrated 
speed of v = 450 m/min which is equivalent to 15 knots, the UAV reaches the next node 
in the list when the allotted time to traverse to the next cell is met. 

WP = {1, 2, 3, 4} 

Let UAV[0] € WP be defined as the current cell of the UAV 


Let UAV[1] € WP be defined as the next node in the list of WP 


Dist(UAV[1]) — Dist(UAV[0] 


v 


Time to next WP = 


Upon the UAV’s arrival to a node, the UAV performs surveillance 
operations as described previously. Upon completion of the observation, the UAV 


continues on to its next location. The user may desire the UAV loiters at each location for 
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a desired time prior to moving on to the next location. SASIO model implementation may 
take this into account to allow for a longer “look” at each node in question. Follow-on 
studies to this thesis may incorporate a dynamic flight profile in which the UAV loiters at 
a location only if detection is made. This allows for additional processing time to discern 
the detection as either a threat or neutral. The next layer of difficulty would include 
allowing a dynamic flight profile for the UAV such that if an object is detected along the 
road, the UAV continues to track this object until a QRF asset can query the object. Once 
the QRF reaches the object, the UAV continues on with the predefined patrol path. 


b. Quick Reaction Force 


(1) Patrolling methods. There are some interesting methods of 
patrolling that SASIO can implement. Due to the modularity of the simulation model, the 
user can define two distinct types of patrols performed by the QRF. Stationary or 
trajectory-based patrols currently exist and each is analyzed in order to provide tactical 
insight to Commanders on strengths and weaknesses of implementing one method over 
another. The QRF patrols the AOI when the report list is empty. The SASIO simulation 
studies two distinct cases of patrol that are currently implemented, stationary and a 


trajectory patrol based on a specific range from the FOB. 


Suppose that a stationary patrol is implemented and the QRF does 
not have the ability to perform observations outside its current location, then the only line 
of defense for protection of the FOB relies on the UAV alone. If the UAV fails to detect 
and identify the threat then the object reaches the FOB unchecked by the QRF. Thus, the 
QRF must be enhanced with the ability to scout the area surrounding the FOB in order to 


provide a second line of defense for SI operations. 


The second method, trajectory based, is a patrol by the QRF in its 
truest sense. Figure 5 illustrates a trajectory patrol of the QRF 750 m away from the FOB. 
The result of performing a patrol under this context may provide undesirable 
consequences that are counter intuitive initially. Suppose that the QRF is performing a 
patrol on the southern leg of the theater, there is a chance that the threat is driving down 


one of the other roads during this time. Based on this instance, the FOB remains 


25 


vulnerable to an attack. However, if the QRF is patrolling the leg by which the threat is 
coming, then the QRF performs the interdiction and thwart any attack prior to the threat 
ever reaching the FOB. 





Figure 5: Illustration of a patrol of one nodes out for the QRF 


The final aspect of the SASIO model is the operation of the QRF 
during an interdiction operation. SASIO implements a Return To FOB (RTB) method 
that utilizes Dijkstra’s algorithm in order to direct the QRF to the FOB. Dijkstra’s 
algorithm is utilized to allow for more complex scenarios. The rational for implementing 


a RTB operation is to emulate the actions that a Commander would expect following 
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interdicting a potential threat rather then continuing on with the patrol. This allows for a 
reset of forces to counteract other threats that are unknown in the area. 

(2) Service time for identification of an object. The QRF 
performs the interdiction of potential threats that were either relayed via the reports 
obtained from the UAV, or by visual detection of these threats by scouting techniques. 
The goal of the QRF is to query and/or interdict the objects at a distance away from the 
FOB in hopes to minimize the number of casualties that would occur in the event that the 
VBIED reached its goal. Since the QRF is a manned asset, the ability to identify and 
service the threats upon a successful interdiction requires time, which is a factor that can 


be varied based on user input. 


The service time for proper identification is what may be varied to 
simulate true interdiction operations. Current implementation allows for this service time 
to be varied in constant time increments from run to run. This parameter provides insight 
into the interdiction operations and an upper limit on the service time to ensure that the 
FOB remains unharmed. By setting the service time to instantaneous, time to service 
equal zero, an upper bound on the number of objects interdicted can be proposed. Future 
revisions of the SASIO model may incorporate a queuing model implementation 


resulting in a nondeterministic service time. 
4. Prediction Model 


Gathering intelligence with respect to the AOI is one of the most vital actions that 
must be performed immediately by the Commander and his personnel. The SASIO model 
implements intelligence gathered for an AOI in the form of a prediction model. This 
prediction model focuses on the motion of the enemy force to include trajectories and 
speed at which each object moves. This prediction model takes the form of four different 
Markov transition matrices depending on the accuracy of the intelligence gathered. Since 
this scenario ensures that all objects travel from an entry node towards the FOB, a 
directed graph is used to represent the trajectories. However, the speed at which the 


objects travel is unknown to the Blue Force. There are three different speeds at which the 
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objects can travel as described under Object Motion Model: 1, 2, and 3 and from these 
speeds, the Blue Force may predict the object’s next location. Figure 7 below depicts how 


the object moves from one node to another. 


Initialize 
t=Vv 









Location A Location B 


While t >0 


Figure 6: Event graph depicting the Object’s motion 


Since the Blue Force intelligence includes the trajectories and the object’s speed, 
then if the Blue Force had tracked the object upon entering the area since the object’s 
motion can be predicted given its current location and speed. However, the UAV is the 
only surveillance asset that the Blue Force has to make observations and the UAV does 
not loiter at the entry nodes; thus, a prediction of where the object is going to be next 
must be made since each observation is a glimpse of a single time. Suppose that the 
object travels every three time steps, and the UAV observes the object, the observation 
could have been made when the object got to that position, or after the object has been 
there for one or two time steps. Since the UAV takes the observation and continues on, a 
report stating that the object is in location X, but the time that the object has been there 
cannot be determined. Therefore, using this example, there is a 7/3 chance that the object 
stays in that location while only a '/; chance the object will move. An example of the 


prediction model for the given example is represented in Figure 7. 
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Figure 7: Prediction Markov transition matrix based on an object transitioning from 
one node to the next adjacent node in three time steps. Recall that node one represents the 
FOB node 


SASIO predicts the motion of the objects by drawing a random number, and if 
that number is less than the probability that the object moves, then the prediction states 
that the object stays in that location. This is iterated each time step, adding each random 
number to itself, until the object is predicted to move. Therefore, if a prediction model is 
based off of intelligence depicting the objects to move every four-time steps, but in 
reality the objects travel every time step, then the objects arrives at the FOB quicker then 


expected and the Blue Force may not interdict the enemy in time. 


Therefore, the parameter of how well the intelligence matches the object’s true 
motion is examined. Through these variations, insight can be gained into the type of 


intelligence obtained and the importance of the quality of that information. 
3, Experimental Design 


The SASIO model can be used to study the effect of many factors on the response 
variables of interest. Experimental design can be used to efficiently explore the design 
space by simultaneously varying the input factors, which can be analyzed by the user 


through the graphical user interface (GUI) built for the SASIO framework. 


The response variables that are explored in this thesis are: number of threat 


objects interdicted, time to interdict all threats, and the distance the threats are interdicted 
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with respect to the FOB. Factor screening is performed first in order to reduce the 
dimensionality of the design space after which an analysis on the remaining factors 


carried out. 
a. Factors, Levels and Justification 


The factors and levels, which are investigated in this thesis, are illustrated 
in Figure 8. Each factor is a unique characteristic of the entities in the simulation and 
each level tries to encapsulate the factors that can change during military operations that 
utilize a UAV for surveillance. Through varying the objects and entities, the model can 
provide insight regarding employment tactics to overcome the scenario threats. The 


objects may be varied as follows: 
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Figure 8: Matrix depicting the factors and levels used for factor screening and 
Design of Experiments 
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Objects 

Velocity — The object’s velocity is modeled as a notional value and can take on 
one of three values and the bounds on these values are described in Figure 8. The object’s 
velocity represents the time required to traverse from one node to the next adjacent node. 
Each value intends to simulate different types of objects whether they are foot traffic (3), 
vehicle on rough terrain (2), or vehicles on paved roads (1). 

Road selection — The objects can be modeled in two different ways with respect to 
the roads selection process. This factor is categorical and can only take on one of two 
values, a single object per road or independent road selection. Modeling the objects as 
one object per road is the simpler technique due to the fact that only a single object 
travels down each road. Upon successful observation and identification of the threat, the 
threat can be interdicted rapidly; therefore minimizing any threat to the FOB. Modeling 
the objects road selection as independent from each other simulates reality since each 
object can travel down any road and is not dependent on the prior object’s selection. 

Time between arrivals — The arrival rates are modeled as a stochastic process 
where the inter-arrival times are exponentially distributed. With a time between arrivals 
of one time step, this represents all objects arriving in the AOI at nearly the same time. 
As the time is increased to five and eleven, the time separating the arrival of one object to 
the next is increased. As the time is increased, it is anticipated that the ease of interdicting 
the threat increases. Therefore with the maximum time used, an upper bound on how well 
the taskforce can perform can be studied 
Agents (UAV) 

Identification — These factors represent the sensor characteristics of the UAV. By 
varying the factors such that p (Rho) is the compliment of y (Gamma) (identification) the 
lower diagonal of a detection or identification matrix may be evaluated. Modeling the 
imperfect sensor produces the same results from the upper versus the lower diagonal and 
therefore only one diagonal must be analyzed. 

Loiter time — This factor represents the amount of time that the UAV remains at a 
single node for detection and identification operations. The levels that studied are a one 


and two time step loiter time. The one time step represents an observation that occurs in 
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one time step prior to moving to the next waypoint while the two time step represents a 
loitering to allow for a more time for an detection and identification operations. 

Patrol radius — The UAV flies predetermined routes for each simulation run. This 
factor explores the variations in the response as the radius increases away from the FOB. 
The radii are measured as Euclidean distance from the FOB and provides insight into 
whether a farther detection range is better than performing more observations at a closer 
distance. 

Agents (QRF) 

Patrol — The QRF’s patrol studied under a stationary context, as well as patrolling 
a single node away from the FOB. These distances provide quick response if a report is 
generated or if a threat is perceived to be close to the FOB. As the patrol distance is 
increased and the threat is traversing toward the FOB via a different road, then the QRF 
must traverse back to the FOB and then interdict the threat. The time that is required for 
the QRF to return to the FOB is additional time that the threat has to reach the FOB. 
Therefore, under these circumstances, there is an increased probability that the threat 
reaches the FOB if the patrol radius is increased dramatically. 

Clearing time — This factor relates to the amount of time that is required to 
identify an object as a threat or not once the object is interdicted. Instantaneous clearing 
time takes a single time step to identify the object. The single-time step and a two-time 
step delay clearing time is evaluated. This time reflects the amount of time required to 
service the object once acquired. As the time is increased, the QRF remains with the 
object longer and therefore may have increased difficulty interdicting another object 
traversing down a different road may reach the FOB. 

Prediction model — The prediction model is based on the perceived velocities of 
the objects through intelligence gathering. As information is gathered on the nominal 
traffic patterns and traffic flows of the objects, a prediction on where the objects are 
given their current location can be made. The prediction model is a Markov transition 
matrix that is studied against the object’s true velocity based on the object motion model 


aforementioned within object velocity. 
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V1 represents an object’s ability to transition from one node to the next adjacent 
node in one time step. V2 and V3 are synonymous with two and three time steps, 
respectively, to transition from one node to the next. The P represents the prediction 
model as described above where P1, P2 and P3 depict an object’s motion based on of 
one, two or three time step transition from one node to the next adjacent node. The 
screening process explores all combinations of the velocities and the prediction models. 
These cross designs illustrate how well the intelligence gathered represents the true 
motion of the objects. The worst-case scenario is the V1xP3 or vise versa model since the 
objects true motion is a single time step to transition while the intelligence states that the 
objects tend to move every three-time steps. Therefore, utilizing the SASIO model, 
analysis on the importance of the intelligence gathered, as modeled by correctness of the 


motion model approximation, can be explored. 
b. Response Variable and Justification 


This thesis explores a number of factors and levels, which are illustrated in 
Figure 8. Through varying the objects and entities, the model can simulate the 
unpredictability of the objects and ultimately provide insight regarding employment 
tactics based on identifying the factors that have the most impact on the response. 


Response variables 


Percentage of threats interdicted — From the clearing list, the number of 
threats interdicted from each design point is obtained and the percentage of threats 
interdicted is calculated based on the number of replications used for each design point. 
Through analysis of this response, the factors based on enemy characteristics, as well as, 


Blue Force asset employment and characteristics can be obtained. 


Time to interdict all threats — From the clearing list, the time that the 
objects were cleared is obtained. This response variable is explored to determine how 
many threats in the AOI are identified and cleared. Through measuring this response, the 


user can obtain better employment of the assets to maximize the number of threats 


obtained. Along with the employment tactics, this response may provide insight into the 
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worst-case settings and necessity to request more assets in order to interdict all of the 


potential threats. 


Distance the threats are interdicted with respect to the FOB — Since the 
purpose of this model is to provide insight into the tactics required to protect the FOB, an 
understanding of the distance that the threats are interdicted with respect to the FOB must 
be studied. Figure 9 illustrates these VBIED threats and their destruction radius varies 
based on the amount of explosives that the threat carries. Supposing that intelligence 
provides the type of vehicles utilized for VBIED based on recent attacks, the user can 
explore and/or predict the maximum the standoff distance obtainable with current assets 
using the SASIO simulation model. The analysis accounts for only the threats that are 
interdicted. 


BATE Explosive Standards 
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Figure 9: Chart illustrating the Vehicle type with respect to the destructive 
capabilities based on the amount of explosives that can be carried, 
http://www.globalsecurity.org/military/intro/images/vbied-standards-chart.jpg 





c Design of Experiments 


Experimental design is the purposeful control of the input factors to the 


experiment to obtain their relationship (if any) with the response variable (Montgomery, 


34 


2009). Through utilizing efficient techniques, information collection for analysis 
purposes can be obtained in the most cost-effective manner. Experimental design allows 
for the control of factors through simulation that would be difficult to control in reality. 
As an example, this simulation allows the programmer to control the rate at which threats 
arrive. In practice, this is an “uncontrollable” factor, but in the context of this simulation, 
it can be systematically changed and thus provide insights that would otherwise be 
unattainable. 

Factor screening is the process of systematically varying the input factors 
in a way that allows the identification of the most influential with respect to the 
responses. The output from the screening provides the analyst with the main effect and 
interactions that are primarily driving the desired response variable. The design matrix 
contains 12 factors that are varied throughout the experiment 

One method of exploring the factor space is through a full-factorial design, 
which is neither efficient nor desired due to the large number of experiments required. 
The number of experiments required to be performed utilizing 12 factors with 2 levels is 
2'* or 4096 design points with zero replications. However, using the factor screening 
technique with fractional factorial design to examine main effects and 2"'-order 
interactions, the number of runs can be reduced to 2“? where k is the number of factors 
and p is the number of generators. Therefore utilizing a fractional factorial design can 


2'*° = 2’ = 128 design points. 


reduce the number of design points to 

Factorial designs are an effective and efficient method for designing 
experiments intended for factor screening. Fractional factorial designs provide a nice 
alternative to running full factorial designs because they provide information on main 
effects and low-order interactions in far fewer experimental tests. Note that linear 
regression is used to analyze the results from factorial designs. The classic factorial 
designs recommended for factor screening utilize only two levels of each input factor — a 


low level and a high level (Montgomery, 2009). 
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Hi. FACTOR SCREENING: NUMERICAL ANALYSIS 


Analysis was performed with respect to each response through a screening design 
and followed by cross-validation based on the influential factors obtained from factor 
screening. Often we are interested in determining what factors affect a particular response 
of interest. In order to accomplish this, we can design an experiment that allows us to 
estimate the magnitude and direction of the factor effects (such as main effects and two 
factor interactions). Factor effects describe numerically how much the response changes 
when each factor is changed. This type of experiment is called a screening experiment 
(and is sometimes referred to as a characterization experiment) (Montgomery, 2009). The 
process of screening the data for the three responses: (1) percentage of threats interdicted, 
(2) time to acquire threats and (3) distance with respect to the FOB that the threats were 
interdicted, is described within this chapter. This chapter discusses the results from the 


screening experiment for each of the three responses. 
A. PERCENTAGE OF THREATS INTERDICTED 


As discussed in Chapter I, counting the number of threats interdicted on a single 
design point and averaging that number based on 1000 replications of that design point, 
the percentage of threats interdicted can be obtained. The percentage of threats 
interdicted is a Bernoulli random variable, which takes the values, zero for not interdicted 
and | for interdicted by Blue Force. This response was transformed via a Log Odds, also 
known as the “logit” function, transformation to ensure the values remain between zero 


and one for the analysis is described below: 


Percentage of threats interdicted 





Transformation = In( )=Y =xBt+e (eqn. 3.1) 


1— Percentage of threats interdicted 


Standard stepwise regression is performed in JMP (JMP, 2009). The linear 


regression model test the following hypothesis: 





H,:4) =f, =&p---= 8, =9, where n is the number of coefficients 
H, : at least one coefficient + 0 (eqn. 3.2) 
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Figure 10: Linear regression model of Log Odds transformation of percentage of 
threats interdicted 


Y Sorted Parameter Estimates 























Term Estimate StdError t Ratio Prob>|t| 
Rho -0.062684 0.004203 -14.92 <.0001* 
Object_Velocity 0.0528697 0.00427 12.38 <.0001* 
Road_Selection{-1)*Patrol{-1] 0.0295146 0.003926 7.52 <.0001* 
Rho*Object_Velocity -0.028155 0.004331 -6.50 <.0001* 
Arrival_Time -0.021945 0.003966 -5.53 <.0001* 
Arrival_Time*Road_Selection[-1] 0.0211875 0.004055 5.22 <.0001* 
Road_Selection{-1] -0.018698 0.003839 -4.87 <.0001* 
Patrol(-1) -0.018574 0.00388  -4.79 <.0001* 
Object_Velocity*Patrol[-1] -0.018242 0.003986 -4.58 <.0001* 
UAV_Velocity*Rho -0.017577 0.004238 -4.15 <.0001* 
Arrival_Time*Patrol|-1] 0.0164039 0.004064 4.04 <.0001* 
UAV_Velocity*Object_Velocity 0.0141289 0.004 —-3.53 0.0005* 
UAV_Velocity 0.0138532 0.004252 _—3.26 0.0013* 
Rho*UAV_Radius 0.013957 0.004295 3.25 0.0014* 
Road_Selection{-1]*Object_Velocity  -0.012697 0.003983 -3.19 0.0017* 
UAV_Velocity*Loiter_Time 0.0127783 0.004025 _—«3.17 0.0018" 
Loiter_Time 0.0120712 0.003894 — 3.10 0.0022* 
UAV_Radius -0.012786 0.004315 -2.96 0.0034* 
Figure 11: Relative significance of each factor based on a linear regression of the Log 


Odds transformation of percentage of threats interdicted. 


Based on an F-statistic (p-value < .0001), shown in Figure 10, the null hypothesis is 
rejected using a .1 percent confidence level; therefore, at least one of the coefficients in 
the model is not equal to zero. Next, the R* value, denoted as Rsquared, has a value of 
0.81 and Adjusted R2, denoted as Rsquared Adj, has a value of 0.79 as shown in Figure 


10. These values represent the amount of variability in the response that is explained by 
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this model and conclude that the model presented is a satisfactory linear model. Since this 
model is used to represent the data for the screening process, the factors that influence 
this response must be obtained. Figure 11 represents the relative weight of each factor 
that influences the response variable. For the purposes of this screening analysis, the top 
20 percent of the total number of factors and interactions is chosen for the cross- 
validation analysis described in Chapter IV. Given the value of R’ is high, the Pareto 
effect, which states that 80% of the results is due to 20% of the input, is the basis for 
choosing only 20% of the significant factors and allows for a minimum number of factors 
used in modeling to correctly and satisfactorily (neither over-fitting nor under-fitting the 


data) describe the data and are as follows: 


e Rho 
° Object velocity 
e Road selection 


e QRF patrol 

e Inter-arrival time of the objects 
Upon completion of analyzing the other responses, these factors, as well as the factors 
from the other responses is aggregated so that cross-validation and prediction analysis 
can be performed. 

Figure 11 shows the relative weightings of each factor. The highest weighting 
factor is Rho, which directly relates to the classification of the threats by the UAV. This 
factor has a negative coefficient and therefore one can conclude that as one decreases this 
false negative identification error rate in the sensors, then the percentage of threats 
interdicted increases. That is, the better the sensor is at identification, then the better the 
chance of properly identifying the threat becomes (decreasing Rho by one unit increases 
the identification by 6%). The next factor is the object’s velocity as the objects take 
longer to traverse from node to node then the probability of interdicting the threat also 
increases. Therefore, if the Blue Force can perturb the speed of the objects in the AOI, 
then there is a higher likelihood of interdicting the threat. The next factor is the 
interaction between road selection and patrol. If the Blue Force can utilize checkpoints to 
ensure that a single object traverses a road per a given timeframe along with performing 


patrols, then the threat is more likely to be obtained. 
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B. TIME TO ACQUIRE THREAT 


The time to acquire the threat is defined as the time between a given threat’s 
arrival into the AOI and its successful interdiction by the QRF. Using this calculated 
value as the response, a best-fit model is constructed based on 1000 replications to 


analyze the influential factors for this response. 


v |» Response Time to acquire threats 
Y Actual by Predicted Plot 





Time to acquire 
threats Actual 
N 
o 








aaa 

25 30 
Time to acquire threats Predicted 
P<.0001 RSq=0.94 RMSE=1.8543 


b oT 
3 10>) 18) (20 


¥ Summary of Fit 
EC 

RSquare (0.938616 
RSquare Adj 0.923747 
Root Mean Square Error ™ 
Mean of Response 7.8005 
Observations (or Sum Wats) 201 

Y Analysis of Variance 

Sum of 

Source DF Squares Mean Square F Ratio 
Model 39 8465.0773 217.053 6 
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C. Total 200 9018.6777 








Figure 12: Linear regression model of the time to acquire threats 


Y Sorted Parameter Estimates 














Term Estimate StdError t Ratio Prob>|t| 
Object_Velocity 6.0259496 0.149718 40.25 <.0001* 
UAV_Radius -1.232851 0.140839 = -8.75 <.0001* 
Arrival_Time*Patrol[-1] 0.8814253 0.144583 —-6.10 <.0001* 
UAV_Radius*Object_Velocity -0.792273 0.145101 -5.46 <.0001* 
Object_Velocity*Patrol{-1] -0.600604 0.144075 4.17 <.0001* 
Road_Selection{-1]*Patrol{-1] 0.5615126 0.141738 3.96 0.0001* 
Figure 13: The relative ordering of factors influencing the time to acquire threats 


Based on an ANOVA hypothesis test, eqn. 3.3, the null hypothesis is rejected 


based on a F-statistic (p < .0001) and using a confidence level of .01%. 





H, : 4, =4,=4,>.-=£f, =9, where n is the number of coefficients 
H, : at least one coefficient # 0 (eqn. 3.3) 
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The R? =0.93 and Adjusted R? = 0.92 indicates that the model explains most of 
the variability of the response. The top 20% factors that influence the response are: 

e Object Velocity 

° UAV Radius 

e QRF Patrol 


e Inter-arrival Time of the Objects 


Based on the factor weighting profile, Figure 13, the most significant factor is 
object velocity. As stated with percentage threats interdicted, if mechanisms such as 
speed bumps or checkpoints are used to perturb the object’s speed, then the time to 
acquire the threat increases. Also, as the UAV radius is decreased, then the time to 
acquire the threat is decreased. This is potentially due to the UAV identifying more 
nearby threats while performing its operation and therefore issuing reports for the QRF to 


interdict. 
C. MEAN DISTANCE AWAY FROM THE FOB 


The mean distance away from the FOB that the threat was interdicted is 
calculated by observing the average distance that the threat was interdicted with respect 
to the FOB for each design point replicated 1000 times. A best-fit model was generated 
using this value as the response variable and the input factors as the regressors. The 


model for this analysis can be seen in Figure 14. 
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Figure 14: Linear regression model for mean distance from FOB 


v Sorted Parameter Estimates 














Term Estimate StdError t Ratio Prob>|t| 
Object_Velocity 0.6499086 0.01561 41.64 <.0001* 
UAV_Radius 0.5071384 0.015844 32.01 <.0001* 
Gamma*Rho 2.6714134 0.136129 19.62 <.0001* 
Gamma 2.6558959 0.137283 19.35 <.0001* 
Rho 2.6471123 0.137175 19.30 <.0001* 
Object_Velocity*Patrol[-1] 0.1185227 0.014317 8.28 <.0001* 
Patrol{-1) 0.1094159 0.014916 7.34 <.0001* 
Arrival_Time*Patrol(-1] -0.099192 0.014614 -6.79 <.0001* 
Road_Selection{-1)*UAV_Radius 0.0945061 0.014652 6.45 <.0001* 
UAV_Radius*Object_Velocity 0.0877332 0.014688 5.97 <.0001* 
Road_Selection[-1)*Patrol[-1] -0.082889 0.014048 -5.90 <.0001* 
Arrival_Time*UAV_Radius 0.0817591 0.015208 5.38 <.0001* 
Road_Selection[-1]*Object_Velocity 0.071135 0.01483 4.80 <.0001* 
UAV_Radius*Patrol[-1) 0.0711642 0.014863 4.79 <.0001* 
Road_Selection[-1) 0.0655551 0.01518 4.32 <.0001* 
Figure 15: The relative ordering of the factors influencing the mean distance away 
from the FOB 


The hypothesis testing for this model is as follows: 





H, :4, =f, =8--= 8, =9, where n is the number of coefficients 
H, : at least one coefficient + 0 (eqn. 3.4) 


As before, the F-statistic (p-value < .0001) was calculated, which means that the 
alternative hypothesis is preferred over the null. By observing the R? =0.97 and the 
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Adjusted R? = 0.96, the conclusion that this model explains most of the variability in the 
response can be made. Therefore, using this model, the following factors are influential in 
the response variable: 

e Object velocity 

° UAV radius 

e Gamma 

e Rho 

e QRF patrol 


By looking at the weighting profile in Figure 15, Object velocity is the most 
dominate factor. Therefore, like the other response variables, if the object’s speed was 
reduced, then the distance that the threat is interdicted is increased. Likewise, if the 
UAV’s radius is increased, the chance to identify the threat at an increased distance 
means that the threat is interdicted by the QRF at an increased distance. Based on these 
findings, the employment of using the UAV at an increased radius is more beneficial to 


maximizing the distance to the FOB then one with a smaller radius. 


D. CONCLUSION 


Based on performing screening for each response, best-fit models were 
constructed. The models for each response had a high R’ value, which suggests adequate 
fits from this analysis, each best-fit model was constructed and adequately modeled the 
response variables with respect to the factor inputs. The aggregated list of factors 
obtained from the above three models are utilized in further cross-validation and 


prediction analysis discussed in Chapter IV. 


The list of factors (and associated responses) was aggregated together so that each 
response is using the same factor set. The reasoning for this is to create a prediction 
model based on all of the factors and to illustrate how each factor contributes to the 
response variables. As seen in Chapter IV, the SASIO model was run with intermediate 
design points related to these factors. Prediction models were then constructed to see how 
well these prediction models correctly represented the new data obtained from additional 


simulation runs. 
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Responses 
Time to 
% Threat acquire Distance 

Factors interdicted threats from FOB 
Object velocity X X X 
UAV radius X X 
Gamma X 
Rho X X 
QRF patrol X X X 
Inter-arrival time of objects X X 
Road selection by the 
objects X 

Table 1: List of influential factors based on the response variables 
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IV. DESIGN OF EXPERIMENTS: CROSS-VALIDATION 


A. OVERVIEW 


New design points were generated based on the findings of the screening exercise. 
The levels within this design include both the high and low levels of the factors, as well 
as intermediate values, which are then transformed into coded units as in the screening 
exercise for cross-validation. The design points (in engineering units) were then 
simulated in the SASIO model in order to explore each response variable in more depth. 
The prediction model formulated in the screening design is tested using cross-validation 


to see how well the prediction model performs. 
B. CROSS-VALIDATION OF PREDICTION DATA 


Based on a fractional factorial design of the input factors, 128 design points were 
created and half were used in cross-validation. Therefore, taking a subset of the 
prediction design points (random selection of 64) with the screening design points, a new 
prediction model was created. Through a visual representation of each response with their 


respective prediction models, analysis on the adequacy of the fit may be performed. 
1. Percentage of Threats Interdicted 


The significant factors for each of the response variables were used in creating the 
prediction models, Figure 16, for the percentage of threats introduced. The 202 
(fractional-factorial design points created by JMP) design points from the screening 
exercise were joined with a random selection of 64 intermediate design points created for 
cross-validation. This set of 266 design points were then used to develop a prediction 
model based on the Log Odds transformation of the percentage of threats interdicted. The 
prediction equation, along with a 95% prediction interval, was then transformed back to 
percentage of threats so that an overlay graph could be created to analyze the percentage 


of threats interdicted. Figure 17 illustrates the prediction model with the remaining design 
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points. Each continuous factor has their associated coefficients, and for the numerical 


factors, the use of the Match statement is used depending on the level utilized. 


4.52 
+ -0.22 * Arrival_Time 
+ 7.48 * Gamma 
+ 6.44 * Rho 
"1" 10.23 
+Match{ Road Selection ||"1" = 0.23 
else >. 
+ -0.16 * UAV_Radius 
+ 0.57 * Object_Velocity 
"1" > 0.13 
+ Match{ Patrol |} "1" > -0.13 
else >. 
+ Gamma * [ Rho * 7.56 | 
"1" 10.17 
+ Rho * Match{ Road_Selection ||"1" > 0.17 
else >. 
"1"\=> 0.12 
+ Rho * Match| Patrol |} "1" > 0.12 
else >. 
"1" (0.14 
"-1" => Match{ Patrof || "1" > -0.14 
else >. 
+ Match/ Road_Selection } "1"3>.0.14 
"1" = Match{ Patrof ||"1" = 0.14 
else >. 
else >. 
"1" >10.21 
+ Object_Velocity* Match| Patrol ||"1" > 0.21 
else >. 
Figure 16: Prediction model equation for Log Odds transformation for percentage of 


threats interdicted 
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Figure 17: 95% prediction model fit for the percentage of threats interdicted with an 
overlay plot of the data obtained from a subset of the sensitivity design points 


Figure 17 illustrates that there are nine data points that fall outside the 95% 
prediction interval, which equate to 14% of the cross-validation data points. Analysis of 
the dominant factor, object velocity, indicates that the data points, which fall above the 
upper prediction interval primarily had the fastest speed, while those that fell below the 
lower limit had the slower speed. From this insight, sub-analysis is performed to create a 
prediction model based on the individual speeds (1, 2 or 3), thereby providing 
recommendation on employment strategies based on the terrain profiles that constrain the 


object’s velocity. 
2 Time to Acquire Threats 


The set of 266 design points, 202 from the screening exercise and 64 from the 
sensitivity design points were used to create a prediction model for the response variable, 
i.e., time to acquire threats. This process of cross-validation was utilized to achieve a 
more accurate prediction model, Figure 18, by incorporating intermediate factor levels 


that were concluded to be significant from the screening design. Based on this linear 
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regression model, the prediction model with a 95% prediction interval was constructed. 
The remaining 64 sensitivity design points were then overlaid on this prediction plot as 


shown in Figure 19. 


16.08 
+ -0.41 * Arrival_Time 
+ -1.58 * Gamma 
+ -1.72* Rho 

"1" > 0.08 
+ Match| Road_Selection ||"1" > 0.08 

else >. 
+ -0.98 * UAV_Radius 
+ 6.28 * Object_Velocity 

F-1"|=> 0.22) 
+ Match{ Patrof || "1" > 0.22 
else >. 
+ Arrival_Time* (Rho * 0.19] 
+ Arrival_Time * | UAV_Radius * -0.28 ] 
"1" > 0.76 
+ Arrival_Time * Match{ Patrol ||'"1" > -0.76 
else >. 
+ Gamma * [ Rho * -1.64] 
+ Rho*| Object_Velocity* -0.21] 
"1" > -0.62 
+ Rho * Match{ Patrol || "1" = 0.62 
else >. 

"21" > UAV_Radius * -0.29 

+Match{ Road Selection ||"1" > UAV_Radius * 0.29 


else >. 
"-1" > 0.52 
"-1" > Match{ Patrol |} "1" = -0.52 
else >. 
+ Match| Road_Selection "1" > 0,52 
"I" = Match{ Pato! || "1" = 0.52 
else >. 
else >. 


+ UAV Radius * { Object_Velocity * -0.68 } 


"1" 10.44 
+ Object_Velocity * Match{ Patrol }|""1" = 0.44 
else >. 





Figure 18: Prediction model equation for time to acquire threats 
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Figure 19: 95% prediction model fit for the time to acquire threats with an overlay 
plot of the data obtained from a subset of the sensitivity design points 


Figure 19 shows all of the design points fall within the prediction interval bounds. 
Based on this graph, the prediction model constructed using the described technique is an 
adequate prediction model. As seen in the graph, there are three distinct levels of the data 
points, a lower, intermediate and an upper level. Each of these levels is due to the 
influence of the object’s velocity. The lower segment correlates to the objects having a 
faster speed while the upper segment is the objects with the slower speed. The reasoning 
is that the objects travel toward the FOB at a faster rate and therefore take a shorter 
amount of time before being interdicted. Likewise, the objects that travel toward the FOB 


at a slower rate are acquired in a longer period of time. 
3. Mean Distance to FOB 


The same techniques used for creating the prediction model for the time to 
acquire the threat response was used for mean distance to FOB. Figure 20 represents the 
prediction model based on the best-fit linear regression model and the subset of cross- 
validation points were overlaid on this prediction model for further analysis illustrated in 


Figure 21. 
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2.14) 
+ 0.075 * Arrival_Time 
+ -0.02 * Rho 

"-1" > 0.08 
+ Match{ Road_Selection ||"1" = -0.08 

else >. 
+ 0.39 * UAV_Radius 
+ 0.6 * Object_Velocity 

"-1" > 0.08 
+ Match{ Patrof||"1" > -0.08 
else >. 
"1" > -0.05 
+ Arrival_Time * Match[ Road_Selection ] "1" >0.05 
else >. 
+ Arrival_Time * [ UAV_Radius * 0.08 | 
+ Arrival_Time * { Object_Velocity * 0.07 | 
"1" > 0.09 
+ Arrival_Time * Match{ Parrot ]|"1" > 0.9 
else >. 
+ Rho * [ UAV_Radius * -0.06 } 
+ Rho * [ Object_Velocity * -0.06 | 
"1" > 0.08 
+ Rho * Match{ Parrot ||"1" > -0.8 
else >. 

"-1" > UAV _Radius * 0.06 
+ Match[ Road_Selection || "1" => UAV_Radius * -0.06 

else >. 

"-1" > Object_Velocity * 0.05 
+ Match{ Road_Selection || "1" => Object_Velocity * -0.05 


else >. 
"1" > -0.08 
"-1" => Match[ Pazrof||"1" > 0.08 
else >. 
+ Match[ Road_Selection | "-1" > 0.08 
"1" => Match{ Pazro/}| "1" > -0.08 
else >. 
else >. 
+ UAV_Radius * [ Object_Velocity * 0.07 | 
"1" > 0.07 
+ UAV_Radius * Match{ Patrol || "1" > -0.07 
else >. 
"1" > 0.08 
+ Object_Velocity * Match{ Patrol ||"1" > -0.08 
else >. 
Figure 20: Prediction model equation for mean distance to FOB 
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Figure 21: 95% prediction model fit for the mean distance to FOB with an overlay 
plot of the data obtained from a subset of the sensitivity design points 


Figure 21 illustrates that there are six data points that fall outside the lower 
prediction interval, which equate to 9% of the 64 sensitivity design points. Further 
investigation of the characteristics of these design points represents the influence of the 
interaction between patrol and object velocity. If the QRF does not patrol, i.e., remains 
stationary at the FOB, and the object’s speed is the slowest, then the resulting data fell 
outside the lower confidence bound. Since the QRF does not perform an interdiction 
unless either a report is issued by the UAV or the threat is within the surveillance region 
of the QRF, then for the six design points, further analysis must be performed to see if the 


results presented in Figure 19 are due to variability in the model. 
4. Conclusion 


The prediction models presented based on a cross-validation technique provided 
adequate models for each of the response variables. Further analysis must be performed 
for percentage of threats (Figure 17) interdicted based on the object’s velocity. Through 
follow-on analysis, an employment strategy that incorporates perturbing the object’s 


velocity may provide an increased efficiency in predicting the interdiction of threats. 


| 


The model that was developed for the time to acquire threats (Figure 19) 
adequately fit the subset of additional simulation experiments. Through the analysis that 
explores the impact of object velocity, better prediction models may be developed. 
Through this analysis, insights regarding each significant factor and relative influence 


may be explored. 


The prediction model (Figure 20) provides insights with respect to the interaction 
between object velocity and the QRF’s patrol employment strategy. The prediction 
interval used an alpha level of 0.05, which means that there is a 5% chance of committing 
a Type I error. Since a 95% prediction interval is created, then the prediction interval 
should encapsulate 95% of the data. Further analysis should either conclude that the 9% 
of the design points that were outside of the lower prediction band for mean distance to 
FOB was either due to noise, or that this influence is significant enough to impact the 
simulation. Based on current analysis, this differentiation cannot be concluded and is 


resolved with additional analysis. 


C. EXPLORATION OF OBJECT’S VELOCITY WITH RESPECT TO EACH 
RESPONSE 


Due to a small percentage of data points falling outside of the prediction intervals, 
further analysis was performed to glean insight into the behavior of the simulation. The 
prediction models were created based on fixing the object’s velocity at one, two and three 
and performing analysis on each of these data sets. A linear regression model was created 
and a prediction model was obtained. A subset of each of the object’s velocity design 
points was reserved to fit with the prediction models. The analysis and insights obtained 


from utilizing this technique follow. 
iF Object Velocity Fixed at the Maximum Speed 


The figures below represent each of the prediction models with their respective 
responses overlaid. Figure 22 shows that all of the data points for the object’s velocity 
fixed at the maximum velocity fall within the 95% prediction interval derived. Some 


useful insights can be obtained from these pictures by observing that there appears to be 
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two distinct data intervals in Figure 22A. This is due to the patrolling characteristics of 
the QRF, the lower data set being the QRF is patrolling and the upper set of data is the 
QRF; remaining at the FOB. Therefore, from this insight, if the AOI cannot be altered to 
reduce the speed of the object, then the QRF should remain at the FOB. Another factor 
that influences the results is the inter-arrival time of the objects. By choosing an AOI that 
minimizes the traffic flow or by establishing checkpoints to minimize the incoming 


objects, then the Blue Force can acquire more threats. 


Figure 22C also shows two distinct data intervals, which is due to whether the 
QRF patrols or not. If the QRF’s employment technique is to patrol, then the standoff 
distance from the FOB is increased; conversely, if the QRF remains at the FOB, then the 
standoff distance is decreased. Therefore, the QRF patrolling affects both the percentage 
of threats interdicted, as well as the mean distance to FOB; however, the benefits of one 


is detrimental to the other. 
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Figure 22: 95% prediction model fit for the (A) percentage of threats interdicted, (B) 
time to acquire threats, (C) mean distance from FOB with an overlay plot of the data 
obtained from a subset of the fixed maximum speed design points 
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Figure 23: Prediction profiler for (A) Log Odds transformation for percentage of 
threats interdicted, (B) time to acquire threats, (C) mean distance from FOB 


Figure 23 illustrates the prediction profiler for each response variable. Figure 23A 


shows that the dominant factor for the percentage of threats interdicted is whether the 
QRF patrols (1) or not (-1). For both the time to acquire (Figure 23B) threats and mean 
distance from FOB (Figure 23C), the dominant factors are object velocity and QRF 
patrol. Based on this analysis, techniques to reduce the object’s velocity increases the 
time to acquire the threats as well as increase the standoff distance from the FOB. Also, 


Figure 23C illustrates that if the QRF patrols, then the standoff distance is maximized. 


Analysis of the model with the objects at their maximum speed, the dominating 


factor is the object’s velocity. To ensure that a large number of threats are interdicted, the 
QRF should remain stationary at the FOB. However, if the commander wants to 


maximize the standoff distance to the FOB, then a patrolling QRF should be employed. 
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Through this analysis, a trade-off exists between interdicting threats or maximizing the 
standoff distance and an understanding of the impact of choosing one over the other must 


be made clear. 
Zz Object Velocity Fixed at the Slowest Speed 


In this study, the SASIO model simulated the design matrix fixing the object’s 
velocity to the minimum value. Taking a subset of this output with the cross-validation 
design matrix, techniques were used to create a linear-regression model. Figure 24, shows 
that all but one data point fall within the prediction model and one can conclude that the 


prediction models are adequate. 


One notes that there are two distinct cases represented in Figure 24A, as with the 
object’s velocity set to the maximum speed. By using the prediction profiler (Figure 25A) 
in conjunction with the prediction plot, the most significant factor is whether the QRF 
patrols or not. The insights obtained from this analysis are that the QRF should remain 
stationary at the FOB to interdict most of the threats. Like the analysis performed for the 
maximum object speed, the AOI should be chosen or checkpoints should be established 


to minimize the traffic flow to the FOB. 


The 95% prediction model (Figure 24B) adequately predicts the data created by 
minimizing the object velocity. The prediction profiler (Figure 25B) demonstrates that 
the most significant factor remains object velocity. This follows that if an object is slow, 
then the time to acquire this threat increases. This is due to the increased time between 


the object entering into the AOI and an interdiction report being made. 


From the analysis performed with respect to the mean distance from FOB, the 
prediction model adequately predicts the data. Two distinct cases arise in this graph and 
they are due to whether the QRF patrols or not. If the QRF remains stationary at the FOB, 
then the mean distance from the FOB is decreased as shown in Figure 25C. However, if 


the QRF patrols, then the standoff distance is maximized. 
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Figure 24: 95% prediction model fit for the (A) percentage of threats interdicted, (B) 
time to acquire threats, (C) mean distance from FOB with an overlay plot of the data 
obtained from a subset of the fixed maximum speed design points 
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Figure 25: Prediction profiler for (A) Log Odds transformation for percentage of 
threats interdicted, (B) time to acquire threats, (C) mean distance from FOB 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 


The factors being used for this analysis should be categorized in two groups: 
factors that are controllable (Blue Force employment techniques) and those that are 
uncontrollable (enemy tactics). Once the factors are broken down into these groups, the 
Blue Force employment techniques can be better explored to maximize the percentage of 
threats interdicted, minimize time to interdict these threats, and maximize the distance 
from the FOB the threats are interdicted, as well as seek methods to best disrupt the 


enemy’s tactics. 
iF Blue Force Employment 


When using the UAV, having sensors with minimal errors results in an increased 
identification rate. This factor has a minor effect on influencing the percentage of threats 
interdicted. This is due to the ability of the UAV to provide an increased identification 
range (i.e., increased standoff identification) and can aid the commander in properly 


responding to a threat prior to its arrival at the FOB. 


The next employment technique is to decide whether the QRF should patrol or 
not. Based on the analysis, the percentage of threats interdicted decreases if the QRF 
patrols; however, the time till interdiction decreases with patrols and the mean distance to 
FOB is maximized as the QRF patrols. The former effect is potentially due to the QRF 
searching a different road while performing its patrol and not being able to perform 
surveillance on the other road. There is a larger decrease in the percentage of threats 
interdicted over the benefits gained in time to interdict or mean distance to FOB. Based 
on having only a single QRF to patrol, one would recommend maintaining the QRF 
stationary until either a report is issued by the UAV or observed by the QRF while in 
close proximity to the FOB. 


of 


Based on the insights provide by the model, the Blue Force can successfully 
perform its operation utilizing a single QRF and UAV. More analysis must be performed 
on the effect of an increase in number of UAVs and QRF units employed. However, 
preliminary consideration of utilizing multiple UAVs lead to recommendations such as to 
employ the UAVs at maximal distance to ensure rapid identification. If there are two 
QRFs, then based on the analysis, the commander should allow one QRF to patrol the 
area, while maintaining one QRF at the FOB. This allows for FOB protection with the 


stationary QRF while an increased interdiction range with the patrolling QRF. 
2. Blue Force Actions External to Current Assets 


The predominate threat factor is the speed at which the object travels down a 
road. As the speed is increased, then all three responses are degraded. Therefore, based 
on this analysis, the Blue Force can set up blockades or other obstacles in order to reduce 


the speed at which all objects traverse down the road. 


The other factor that the Blue Force can influence is the road selection of the 
objects. If each road is limited to travelling down a single road, then each response 
increases in the Blue favor. Setting up checkpoints prior to the entrance into the AOI can 
counteract this problem. These checkpoints can be provided by the indigenous population 
that can provide intelligence to the Blue Force, but whose primary goal is to perturb the 


flow of traffic down each road. 
B. ONGOING RESEARCH 


Ongoing research, which utilize SASIO model framework, studies the potential 
improvement in effectiveness by increasing the available surveillance and interdiction 
assets. Another study utilizing the SASIO model framework is a game-theoretic approach 
that encompasses an adversarial threat and potential strategies to combat these threats. 
The study explores current Blue Force strategies and compares those strategies with 


possible improvements based on an effective use of surveillance and interdiction assets. 


The current SASIO model implementation requires either utilizing command-line 


arguments or a graphical user interface (GUI) to run the program. Current work is to 
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continue to evolve the GUI interface as new concept and/or ideas arise such as: switching 
from a screening design to sensitivity analysis, implementing the desired number of 
replications, and choosing the scenario based on user inputs. As seen in Figure 26, the 
GUI is currently used for the SASIO scenario posed in this thesis; however, the evolution 


of this program allows an easy to command shell to execute tactical scenarios. 
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Figure 26: SASIO tactical installation protection graphical user interface 
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